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(Note: An appendix with definitions of terms used is attached to this report.) 
I. INTRODUCTION 

The Accountability System Review Panel (Panel) was created by a Memorandum of 
Understanding (MOU) entered into by the Governor, the Speaker of the House, the 
President Pro Tempore of the Senate, and the State Superintendent of Public 
Instruction. 

The MOU established the Panel to carry out the following duties: 

1. Make recommendations regarding the A-F accountability system, including 
recommendations regarding measurements based on individual academic 
performance and growth to proficiency and avoiding recommendations based on 
measurement of student performance or growth compared with peers. 

2. Consider a wide range of data in making its recommendations. 

3. Examine other states' accountability systems to look for innovative solutions. 

4. Ensure the fairness of any recommended accountability system. 

5. Compose a final report with recommendations no later than November 1, 
2013. 

6. Exist until after the deadline for such report until December 31 , 2013, for the 
purpose of receiving and investigating any clarifying questions posed by the 
State Board of Education, the Indiana Department of Education, the Governor, 
the House, or the Senate, unless otherwise extended or disbanded by the terms 
of the MOU. 

Each signatory to the MOU appointed four members: one teacher, one principal, one 
superintendent, and one technical advisor. In addition, the State Superintendent of 
Public Instruction, Glenda Ritz, served as a member and Co-Chairperson. Dr. Steve 
Yager, Superintendent of Southwest Allen County Schools, served as the other Co- 
Chairperson. 

Under Indiana's current school accountability system, schools are assessed for 
performance under two standard models - an elementary and middle school model and 
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a high school model. (There is an allowance made for schools that do not conform to a 
traditional model, such as a combined school.) The elementary and middle school 
model measures student passage rates on state-wide English/language arts and math 
tests; it also rewards schools for student growth. The high school model includes four 
measures: 

(1) Passage rates on 10th grade end of course assessments (ECA) in English 10 
and Algebra 1, as well as student growth in these subjects. 

(2) Passage rates on both ECA tests by students who initially did not pass these 
tests in 10th grade, but pass before graduation. 

(3) High school graduation rates. 

(4) College and career readiness (CCR) based on student achievement of the 
following indicators: (1) A passing score on at least one Advanced Placement 
(AP) examination. (2) A passing score on at least one International 
Baccalaureate (IB) examination. (3) The completion of at least three college 
credit hours through a dual credit course. (4) The receipt of an industry 
certification. 

(Note: A brief explanation of the how school scores are determined under the current 
school accountability model is included as Appendix B.) 

In developing a new or revised school accountability system, the Panel followed IC 20- 
31-8-3, as amended by HEA 1427-2013, which requires the State Board of Education 
(State Board) to "establish a number of categories, using an "A" through "F" grading 
scale, to designate performance based on the individual student academic performance 
and growth to proficiency in each school.". In addition, IC 20-31 -8-1 (a) provides that 
"The performance of a school's students on the ISTEP program test and other 
assessments recommended by the education roundtable and approved by the state 
board are the primary and majority means of assessing a school's improvement.". 

Indiana has a waiver from the requirements of the federal "No Child Left Behind" 
statutes that requires certain elements in the state's school accountability system, 
including the following: 

- The system must look at student achievement for all defined subgroups of 
students in at least reading/language arts and math, graduation rates, and 
school performance and progress over time. 

- Once the state has adopted a high-quality assessment, it must take into 
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account student growth for all subgroups. A state must report both its pass rate 
and participation rate on the assessment. 

- Set new ambitious but achievable measurable objectives for all subgroups in at 
least reading/language arts and math. 

- Provide incentives and recognition for success, including, if possible, rewarding 
Title I schools making the most progress and identifying the schools as Title I 
"reward schools". 

- Effect dramatic, systematic change in the lowest-performing schools, identifying 
the schools as Title I "priority schools" and ensuring meaningful interventions. 

- Work to close achievement gaps by identifying schools with the greatest 
achievement gaps as Title I "focus schools" and ensuring interventions based on 
reviews of the specific academic needs of the school and its students. 

- Provide incentives and supports to ensure continuous improvement in Title I 
schools that are not making progress in improving student achievement and 
narrowing achievement gaps. 

- Build capacity to improve student learning in all schools. 

Under the current federal waiver, for accountability purposes, Indiana reports data for 
two "super" subgroups of students - the top 75% and the bottom 25% - instead of data 
for ten subgroups of students that would be required in the absence of the waiver. The 
data for the ten subgroups are reported for monitoring purposes. Indiana's waiver 
expires in 2014; it is likely that Indiana will apply for a new waiver. 

II. SUMMARY OF WORK PROGRAM 

The Panel met seven times before November 1, 2013. 

September 19, 2013: The Panel received information on the Indiana Open Door Law 
and the MOU. A historical perspective on Indiana's accountability system was 
presented, as well as information concerning federal and state legal requirements for 
accountability systems and Indiana's waiver from certain federal accountability 
requirements. The Panel began discussion concerning what elements the members 
would like to see included in a system as well as elements that the members would not 
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like to include in a system, taking into account policy needs, development needs, and 
implementation. 

September 24, 2013: The Panel began ranking the elements to be included in an 
accountability system. The Panel received information concerning the role of 
assessments in accountability systems, particularly in models that focus on growth to 
proficiency, which is required under Indiana statute. The Panel began examining three 
existing growth models of accountability: the gain, the categorical, and the trajectory, 
and discussed the components of each model, as well as components Indiana's model 
should include. 

October 4, 2013: The Panel considered transition options for going from Indiana's 
current accountability system to a new system, and held considerable discussion of the 
gain, trajectory, categorical, and student growth percentile models, including 
considering other states' accountability systems. There was agreement that Indiana's 
model should look at a student's growth or lack of growth over the course of a year. A 
hybrid growth model, combining elements of the trajectory model and a criterion- 
referenced categorical model, was discussed. 

October 8, 2013: The Panel received and discussed information concerning the current 
high school achievement model, and information on multiple measures of achievement. 
Several members of the Panel expressed concerns with the penalty aspects of the 
current high school model, and considered student data currently collected in Indiana 
that could be used as measures of achievement, as well as data could potentially be 
collected. The Panel received worksheets for developing an accountability framework to 
determine performance indicators and the weight to be given to specific indicators, and 
looked at models developed by Panel members based upon discussions held at the 
previous meetings. 

October 18, 2013: The Panel received information on reading assessments for 
determining growth. The Panel reviewed options for frameworks and components for 
accountability system models, and discussed whether existing student data can be 
analyzed in a timely fashion to test the feasibility of the Panel's recommendations. The 
members reached a consensus to have the accountability grading system based on a 
100 point scale instead of the current four point scale, and to have different frameworks 
for elementary/middle grades and high school grades. 

October 24, 2013: The Panel worked through a number of questions concerning 
elements to be included in the accountability system model, and came to a consensus 
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concerning the following: 

- To include a trajectory component in the growth domain of the model to satisfy 
the statutory requirement of determining "growth to proficiency". 

- A categorical element should not be included in the performance domain of the 
model. 

- Categorical improvement in growth in high school should be a part of the final 
model, but the current use of improvement for grades 8 through 10 should be 
continued until new assessments that support the final system are in place. 

- The current method of determining improvement for grades 10 through 12 
should be used until new assessments are adopted, after which improvement 
between the grade in which the assessment is administered and grade 12 should 
be rewarded. 

- As a performance indicator, the current system of awarding points should be 
transitioned through the use of a multiplier over the course of several years to a 
system in which the percentage of students who have attained a CCR indicator 
is used. For growth in CCR indicators, the increase in the number of students 
who have CCR indicators in grade 10 to the number of students who have CCR 
indicators in grade 12 should be used. 

- To continue to use categorical scores as measures of growth for the two super 
subgroups, rather than using the ten subgroups. 

- Weights for the domains of performance and growth: in the high school portion 
of the accountability system, the weights should be 70% performance and 30% 
growth. 

The Panel discussed, but did not reach a consensus concerning, the use of the terms 
"commendable" and "proficient" for Title 1 schools that receive B and C grades, 
respectively. 

October 28, 2013: The Panel considered and took action on the following issues: 

- For super subgroups under the federal NCLB waiver, for which Indiana's 
current waiver uses top 75% - bottom 25% subgroups, Utah and Louisiana use 
proficient and nonproficient students as their subgroups instead of set 
percentages. No action was taken on the issue of changing the subgroups for 
the next waiver application. 

- Charts of school report cards, with weights arbitrarily determined for 
demonstration purposes, calculated using student data from 2012, were 
distributed to show how the new model will impact existing A, C, and F graded 
schools. The Panel decided that for participation rate factor, if the percentage of 
participation is 95% or higher, the school will receive a full credit; if the 
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percentage is 94.9% or lower, the school will receive a partial credit equal to the 
percentage. Schools will not be penalized for less than 95% participation, and 
the elementary and high school models will treat participation in the same 
manner. 

- Growth to proficiency: how it will be determined, whether a better way of stating 
"growth to proficiency" for a school is "students meeting targeted proficiency", 
the necessity of having a statistical analysis to determine appropriate time 
frames for students to move to proficiency, and whether a categorical model is 
sufficient to determine growth to proficiency. The Panel decided that the term 
"targeted proficiency" would be used. 

- Unique school situations: Under the current model, small schools (with classes 
below the size required for the model) are graded by going back up to three 
years to establish a cohort of a sufficient size. The Panel recommended that 
practice be continued. For new schools, the Panel had no recommendations, as 
the model will accommodate new schools. For dropout recovery schools, the 
changes made to the high school model should also be made for dropout 
recovery schools during the transition period. For school configurations with no 
tested grades (for example, a K-2 school or a grade 9 only school): while 
eventually grades 1-11 will be tested in at least some areas, allowing specific 
grades to be assigned, during the transition, the "feeder" school model, in which 
the school receives the grade of the school into which it sends students, will be 
continued. 

- Anomalies in grades assigned: DOE will develop flags to identify anomalies in 
the accountability system's assignation of grades to schools. 

The Panel reviewed the draft final report (Exhibit C), and made changes, including a 
change to the weights given performance and growth in the high school model, and 
corrections to the report. On the final report, Mr. Redelman raised concerns about the 
lack of data in evaluating recommendations for a model and the lack of agreement on a 
matrix for the categorical model, as well as questioning the definition of "targeted 
growth" added to the report. The final report was adopted on a voice vote of 16-1 . 
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III. MATERIALS CONSIDERED 



The Panel developed the following framework of values for an accountability system: 

1. Growth for all students is highly valued and schools should be rewarded for 
individual student growth. 

2. The model should be clear, understandable, fair, and transparent. Schools 
should be able to understand the statistical calculations and be able to use the 
data to inform instruction. 

3. Multiple data points should inform both growth and performance. 

4. The model should allow for flexibility for changes in assessments, allow for all 
configurations of schools, and align with federal Title I category requirements. 

The Panel had access to the Council of Chief School Officers report of accountability 
system comparisons across 38 states that have a No Child Left Behind flexibility waiver. 
Among these states, the elements of the following states' systems were adopted by the 
Panel: 

- Colorado (the addition of a trajectory model within Indiana's current growth 
component to indicate "catch up, keep up, move up" once a baseline model has 
been created to calculate growth scores). 

- Alaska & Illinois (expand Indiana's current categorical model from 3 categories 
to a minimum of 5 categories and to allow for school points for individual student 
growth progression from category to category) 

- Alaska and other states (use easy to understand 100-point scale) 

A Practitioner's Guide to Growth Models by Andrew D. Ho, Harvard Graduate School of 
Education and Katherine E. Castellano, University of California, was used to expand the 
Panel's knowledge base and determine effective yet simple means to measuring 
growth. The Panel determined that Indiana should show student growth using both 
categorical and trajectory approaches. While Indiana's current system shows minimal 
student growth across 3 levels (Do Not Pass, Pass, and Pass +), the Panel decided to 
fully develop the categorical portion by delineating at least 5 categories within the 3 
levels for the purpose of awarding growth points for individual student growth crossing 
categories. In addition, in order to meet the requirements of HEA 1427-2013 to show 
growth to proficiency, the Panel decided that the trajectory approach already 
established within Indiana's current growth model should be revised to reflect a criterion 
approach as opposed to the current percentile approach, which does not comply with 
the requirement of IC 20-31-8-3 to be based on individual student performance. 
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Recommendations from the report "The Examination of Indiana's A to F School 
Accountability Model", September 6, 2013, by John Grew and William Sheldrake, also 
served as a catalyst for considering multiple data points for accountability in addition to 
ISTEP testing. 

The Panel reviewed the following data points for consideration within the two domains 
of performance and growth: 



Data Points 


Accepted by Panel? 


Performance or Growth? 


Math Performance (1-10) 


YES 


P 


Math Growth (2-12) 


YES 


G 


Math Participation 


YES 


P 


ELA Performance (1-10) 


YES 


P 


ELA Growth (2-12) 


YES 


G 


ELA Participation 


YES 


P 


Science Performance 


NO 




Science Participation 


NO 




Reading Performance (1- 
11) 


YES 


P 


Reading Growth (2-12) 


YES 


G 


CCR Performance 


YES 


P 


CCR Growth 


YES 


G 


Graduation Rate 


YES 


P 


Graduation Growth 


YES 


G 


Attendance 


NO 




Suspension/Expulsion 
Rate 


NO 





8 



Classroom size, bullying 


NO 




rate, student engagement, 






principal and teacher 






effectiveness, parent 






engagement, student, 






career employment "soft 






skills" 
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The Panel recommends the following framework for the school accountability 
model, subject to validation by statistical analysis as data becomes available. The 
Panel recognizes that it may be asked to conduct follow-up recommendations in 
addition to the work included in this report. The Panel recognizes that work 
conducted for additional recommendations will include more extensive use of 
subject matter experts as the statistical aspect of the accountability system is 
realized. 

IV. SCHOOL ACCOUNTABILITY SYSTEM RECOMMENDATIONS 

The Panel recommends the following interdependent components for the Indiana 
school accountability system: 

(1) The grading scale for the A - F system, currently a 4-point scale, will be changed to 
a 100-point scale. 

(2) The accountability system model will have different frameworks for grades 1-8 and 
grades 9-12. 

(3) The accountability system will have two domains: performance and growth. 

(4) The model will allow for changes in assessments, including any new assessments 
that may be selected once CCR standards are adopted as required under HEA 1427- 
2013. 

(5) As required under IC 20-31-8-1, the performance of a school's students on the 
ISTEP program test and other assessments recommended by the Education 
Roundtable and approved by the State Board are the primary and majority means of 
assessing a school's improvement. 

(6) The model will include the data points to measure reading growth and performance 
in grades 1-10 (possibly to grade 11), when data becomes available. 

(7) The model will measure CCR indicators in both domains of performance and 
growth. 

(8) The CCR indicators will include the PSAT as a data point. 

(9) The model will measure targeted growth. 
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(10) The targeted growth for each student will be determined annually. 

(11) The model will measure categorical growth improvement. 

(12) The model will allow targeted growth to be measured for high school when data 
becomes available following the adoption by the State Board of new assessments that 
enable the development of a vertical scale. 

(13) The model will use improvement rates as data points for growth in the 10th to 12th 
grade. 

(14) The model will retain the CCR goal at 25% student attainment (the current level) 
and the data will be multiplied by 4 to create points. The model will allow for an increase 
in the significance of the CCR goal. 

(1 5) The model will use a categorical improvement indicator for the super subgroups in 
the growth domain. 

(16) Title I category descriptors will be aligned with the model by identifying terms that 
align with A - F categories of the accountability system; however, the Panel makes no 
recommendation concerning what the terms should be. 

(17) The model will be developed to have vertical scale alignment with assessments in 
grades 1-10 (possibly grade 11). 

(18) The model will expand to at least 5 performance categories that are delineated 
within the current 3 performance levels to show improvement in growth. 
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The Panel recommends the following implementation framework for the school 
accountability model, subject to validation by statistical analysis as data 
becomes available. The Panel recognizes that it may be asked to conduct 
follow-up recommendations in addition to the work included in this report. The 
Panel recognizes that work conducted for additional recommendations will 
include more extensive use of subject matter experts as the statistical aspect of 
the accountability system is realized. 

V. ACCOUNTABILITY MODEL - IMPLEMENTATION RECOMMENDATIONS 

1. The complete recommended accountability model should be used to assess all 
schools with tested grades starting in school year 2014-15. 

2. Point scale. 

a. The model should use a 0.0 to 100.0 scale. 

b. Category placements are established based on total points (weighted average 
of domain points) assigned to a school using the following scale: 



3. Total points assigned to a school should be a weighted average of the designated 
domains within the accountability framework. 

A. Performance 



i. The performance domain will be assigned a weight in the overall 
framework. 

ii. Total performance points will be the sum of the domain indicators final 
points. 

iii. Domain placements are established based on domain points (weighted 
average of indicator points) assigned using the following scale: 

90.0 to 100.0 A 
80.0 to 89.9 B 



90.0 to 100.0 
80.0 to 89.9 
70.0 to 79.9 
60.0 to 69.9 
0.0 to 59.9 



A 
B 
C 
D 
F 
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70.0 to 79.9 
60.0 to 69.9 
0.0 to 59.9 



C 
D 
F 



iv. Indicators to be included are: 



1. English/Language Arts 



a. Points are to be assigned for grades 1-10 (or possibly 
grade 1 1 ) where test data are available. 

b. Points are not awarded for grade 12. 

c. Points awarded in each grade span should equal the 
product of the state assessment pass rate and the 
participation rate factor. 

i. If participation rate is greater than or equal to 95%, 
then the participation rate factor should be 1. 

ii. If participation rate is less than 95%, then the 
participation rate factors should equal the 
participation rate in decimal form. 

d. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

e. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 



a. Points are to be assigned for grades 1-10 (or possibly 
grade 1 1 ) where test data are available. 

b. Points are not awarded for grade 12 

c. Points awarded in each grade span should equal the 
product of the state assessment pass rate and the 
participation rate factor. 

i. If participation rate is greater than or equal to 95%, 
then the participation rate factor should be 1. 

ii. If participation rate is less than 95%, then the 
participation rate factors should equal the 
participation rate in decimal form. 

d. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

e. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 



2. Math 
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3. Reading 

a. Points are to be assigned for grades 1-10 (or possibly 
grade 1 1 ) where test data are available. 

b. Points are not awarded for grade 12 

c. Points awarded in each grade span should equal the 
product of the state assessment pass rate and the 
participation rate factor. 

i. If participation rate is greater than or equal to 95%, 
then the participation rate factor should be 1. 

ii. If participation rate is less than 95%, then the 
participation rate factors should equal the 
participation rate in decimal form. 

d. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

e. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 

4. College and Career Readiness Achievement 

a. Points are to be assigned for grade 12. 

b. Points are not awarded for grades 1-11. 

c. Points awarded in each grade span should equal the 
product of the CCR rate and the state readiness factor. The 
state readiness factor should be determined through the 
following: 

i. The readiness factor should be the quotient of the 
total achievable and the annual goal. Currently, the 
formula would be 100/25=4. 

ii. The current goal presented to Indiana schools is 
25%. The accountability panel recommends review of 
the current goal including recommendations from the 
Department of Workforce Development and the 
Commission for Higher Education as to an attainable 
goal and additional accurate measures of CCR. 

d. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

e. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 

5. Graduation 
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a. Points are to be assigned for grade 12. 

b. Points are not awarded for grades 1-11. 

c. Points awarded in each grade span should equal the 
graduation rate. 

i. If graduation rate is greater than or equal to 90%, 
then the score should be 100. 

ii. If graduation rate is less than 90%, then the score 
should equal the graduation rate in percent form. 

d. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

e. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 



i. The growth domain will be assigned a weight in the overall framework. 

ii. Total growth points will be the sum of the domain indicators final points. 

iii. Domain placements are established based on domain points (weighted 
average of indicator points) assigned using the following scale: 



a. Points are to be assigned for grades 2-12 where test data 
is available. 

b. Points awarded in each grade span should be assigned 
as follows: 

i. Grades 2-11 points should be the average of three 

growth indicators: 

1. For students meeting targeted 
growth, schools should be awarded 
points equal to the percent of students 
achieving the expected annual growth. 



B. Growth 



90.0 to 100.0 
80.0 to 89.9 
70.0 to 79.9 
60.0 to 69.9 
0.0 to 59.9 



A 
B 
C 
D 
F 



iv. Indicators to be included are: 



1. English/Language Arts 
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a. Currently data is only available 
for grades 3-8. 

b. In absence of continuous data 
on a vertical scale, growth for the 
indicator should be measured as 
improvement from grade 8 to 
grade 10 using a simplified 
category status improvement 
calculation (3x3 improvement 
scale). 

2. Higher performing categorical growth 
improvement points should be awarded 
equal to the average categorical score 
for students within the top 75% of prior 
year performance. 

3. Lower performing categorical growth 
Improvement points should be awarded 
equal to the average categorical score 
for students within the bottom 25% of 
prior year performance. 

ii. Grade 12 points should be the awarded equal to 
the rate of improvement of students on graduation 
qualifying exams between the primary administration 
year and graduation. 

c. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

d. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 

2. Math 

a. Points are to be assigned for grades 2-12 where test data 
is available. 

b. Points awarded in each grade span should be assigned 
as follows: 

i. Grades 2-11 points should be the average of three 

growth indicators: 

1. For students meeting targeted growth, 
schools should be awarded points equal to the 
percent of students achieving the expected 
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annual growth. 

a. Currently data is only available for 
grades 3-8. 

b. In absence of continuous data on a 
vertical scale, growth for the indicator 
should be measured as improvement 
from grade 8 to grade 10 using a 
simplified category status improvement 
calculation (3x3 improvement scale). 

2. Higher performing categorical growth 
improvement points should be awarded equal 
to the average categorical score for students 
within the top 75% of prior year performance. 

3. Lower performing categorical growth 
Improvement points should be awarded equal 
to the average categorical score for students 
within the bottom 25% of prior year 
performance. 

ii. Grade 12 points should be the awarded equal to 
the rate of improvement of students on graduation 
qualifying exams between the primary administration 
year and graduation 

c. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

d. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 



3. Reading 

a. Points are to be assigned for grades 2-1 1 where test data 
is available. 

b. Points awarded in each grade span should be assigned 
as follows: 

i. Grades 2-11 points should be the average of three 

growth indicators: 

1. For students meeting targeted growth, 
schools should be awarded points equal to the 
percent of students achieving the expected 
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annual growth using a simplified category 
status improvement calculation (3x3 
improvement scale. 

2. Higher performing categorical growth 
improvement points should be awarded equal 
to the average categorical score for students 
within the top 75% of prior year performance. 

3. Lower performing categorical growth 
improvement points should be awarded equal 
to the average categorical score for students 
within the bottom 25% of prior year 
performance. 

ii. Grade 12, no points are awarded. 

c. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

d. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 

4. College and Career Readiness Achievement 

a. Points are to be assigned for grade 12. 

b. Points are not awarded for grades 1-11. 

c. Points awarded in each grade span should equal the rate 
in which graduates improved CCR status from 
non-achievement by the end of the 10th grade year to 
achievement by graduation. 

d. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

e. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 

5. Graduation 

a. Points are to be assigned for grade 12. 

b. Points are not awarded for grades 1-11. 

c. Points awarded in each grade span should be assigned 
as follows: 

i. The improvement rate should equal the quotient of 
the current cohort graduates plus additional out of 
cohort graduates for the accountable year and the 
total cohort. 
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ii. The Panel recommends statistical analysis 
concerning graduation improvement to ensure out of 
cohort graduates are captured in the improvement 
rate calculation. 

d. Overall points should be the sum all applicable grade 
span points weighted to reflect enrollment in each span. 

e. Overall final points for the indicator should be the product 
of the indicator points and the indicator weighting. 

4. The accountability model may use the following weights to determine final points 
assigned to a school, pending statistical analysis: 

A. For grades 1 to 8: 

i. The Performance domain should receive less weight than the Growth 
domain. 

1. Assessment indicators should be weighted as available to equal 
100%: 

a. English/Language arts. 

b. Math. 

c. Reading. 

2. Other indicators. 

ii. The Growth domain should receive more weight than the Performance 
domain. 

1. Content area growth indicators should be weighted as available 
to equal 100%: 

a. English/Language arts. 

b. Math. 

c. Reading. 

2. Other indicators. 

3. Growth points: 

a. For students meeting targeted growth, weighting should 
be 10%. 

b. For students achieving above targeted growth, weighting 
should be 45%. 

c. For students achieving below targeted growth, weighting 
should be 45%. 
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B. For grades 9 to 12: 

i. The Performance domain should receive more weight than the Growth 
domain. 

1. Assessment indicators should be weighted as available to equal 
40%: 

a. English/Language arts. 

b. Math. 

c. Reading. 

2. CCR should be weighted 30%. 

3. Graduation rate should be weighted 30%. 

ii. The Growth domain should receive less weight than the Performance 
domain. 

1. Content area growth indicators should be weighted as available 
to equal 40%: 

a. English/Language arts. 

b. Math. 

c. Reading. 

2. CCR should be weighted 30%. 

3. Graduation rate should be weighted 30%. 

4. Growth points: 

a. For students meeting targeted growth, weighting should 
be 10%. 

b. For students achieving above targeted growth, weighting 
should be 45%. 

c. For students achieving below targeted growth, weighting 
should be 45%. 

5. For unique school situations, the accountability model should use the following 
recommendations: 

A. Small schools (with classes below the size required for the model): The 
current practice of going back up to three years to establish a cohort of a 
sufficient size to determine a grade should be continued. 

B. New schools: The Panel has no recommendation, as the model will 
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accommodate new schools. 

C. Dropout recovery schools: The changes made to the high school model 
during the transition period should also be made for dropout recovery schools 
during the transition period. In January, 2013, the State Board initiated 
rulemaking to adopt alternative metrics for these schools. 

D. School configurations with no tested grades (for example, a K-2 school or a 
grade 9 only school): Until test scores are available for these grades, the 
"feeder" school model should be continued. 

6. Perceived anomalies in grades assigned to schools: DOE will develop flags to 
identify anomalies in the accountability system's assignment of grades to schools. 
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The Panel recommends the following transitional elements for the school 
accountability model, subject to validation by statistical analysis as data 
becomes available. The Panel recognizes that it may be asked to conduct 
follow-up recommendations in addition to the work included in this report. The 
Panel recognizes that work conducted for additional recommendations will 
include more extensive use of subject matter experts as the statistical aspect of 
the accountability system is realized. 

VI. RECOMMENDATIONS FOR TRANSITION BETWEEN THE CURRENT SCHOOL 
ACCOUNTABILITY SYSTEM AND THE NEW SCHOOL ACCOUNTABILITY SYSTEM 

The Panel recommends full implementation of the new school accountability system in 
2014-15. This assumes a transitional period as the system will need to be adjusted, 
assuming a new assessment is adopted that allows for a vertical scale through grade 
10 or grade 11. 

The Panel recommends that the model should be validated by various methods, 
including beta testing, during the rules development process to insure accuracy of the 
accountability system. Validation should use actual data from past years. 

The Panel recommends that procedures be developed in rule that automatically place a 
school in a review process if the overall grade changes by two or more grades in one 
year. One of the concerns of transitioning to a new accountability system is to buffer 
schools from significant changes in category placement until the accountability model is 
mature. Although schools may be experiencing large changes in their performance or 
growth, or both, due to their own actions, safeguards should be in put in place to protect 
schools from unforeseen deficiencies in the model. 

The Panel recommends that protocols and procedures should be developed for 
addressing any grade configurations issues when not adequately addressed by rule. 
Although the model presented in the Panel's recommendation factors in different 
current configurations, past experience shows that it is difficult to insure rules for the 
accountability system will address all configuration complexities. 

The Panel recommends a web-based "calculator" be developed for local school 
administrator use. School administrators should be able to replicate and explain how 
their schools' grades were calculated. 

The Panel recommends that the model developed, and proposed changes to the 
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model, be communicated to schools and school corporations as quickly as possible. 



The Panel recognizes that the model developed will be subject to federal peer review 
processes, and that new annual measurable objectives must be written. 

The Panel recommends that steps for an appeal process for the overall category 
placement be simple and clear. The Panel recommends that schools receive a written 
notice of appeal findings. The written notice should include the reason for the findings 
and whether the remedy affects the overall category placement. 

Grew/Sheldrake Report Transition Recommendations 

The Panel is in agreement with the following recommendations from the 
Grew/Sheldrake report, "Examination of Indiana's A to F School Accountability Model", 
September 6, 2013, relevant to the Panel's scope of duties: 

Developing a Revised Accountability System Under HEA 1427-2013: The 
authors observe and recommend: 

The authors observe that the recently announced memorandum of 
understanding between the Governor, the General Assembly, and the 
Superintendent for establishing a collaborative process for development of 
a new accountability rule is an excellent step towards increasing support 
by the educational community and the public. 

The process of development of a new system should: 

1. Provide for extensive involvement by experts and practitioners 
from the education community. 

2. Provide for transparency in all decision-making. 

3. Result in development of a new system that is as simple as 
possible, more easily understood, and equitable. 

In compliance with HEA 1427 - 2013, the new accountability system 
should incorporate measures that involve less reliance on standardized 
tests passage rates and more reliance on individual student growth based 
on criterion-referenced measures. 

Further Recommendations regarding the Revised Accountability System: 
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Additional measures for the elementary/middle school model should be 
included, besides the two student test measures, which provide additional 
indicators of school performance. 

Because of the complexity involved in implementing any new 
accountability system, the system should be piloted prior to 
implementation, if possible, permitting the Department of Education to 
solicit and receive extensive feedback from schools, adequately perform 
programming tests, and evaluate policy components incorporated into the 
system. 

In order to ensure that the General Assembly has the capability to perform 
analyses on the new accountability system, Legislative Services Agency 
staff should be provided with ongoing access to all data and computer 
programming necessary for the Agency to replicate results and respond to 
various inquiries from legislators about the system.* 

* The relationship between DOE and the Legislative Services Agency will be 
determined through an MOU. 
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APPENDIX A 



DEFINITIONS (AS USED IN THIS REPORT) 

Categorical model: Evaluates students moving from one performance category to 
another. Requires the use of cut scores. 

College and Career Readiness (CCR): For a high school, measures of student college 
and career readiness include the number of students attaining International 
Baccalaureate degrees, successfully completing Advanced Placement courses, 
successfully completing dual credit courses, receiving industry certifications, or attaining 
satisfactory scores on PSAT exams. For a student, successfully achieving one or more 
of the CCR components. 

Criterion referenced: A test in which an individual's performance is compared to a 
performance standard and not to the performance of other individuals in a peer group. 

Cut score: A score used to determine the minimum competency level needed to pass 
a test. 

Graduation rate: The percentage of students within a cohort who graduate during their 
expected graduation year. (The cohort is the class of students who are considered to 
have entered grade 9 in the same year and expected to graduate three years after 
entering grade 9.) 

Growth to proficiency: A student's progress to meet and pass established proficiency 
levels, as demonstrated by at least two data points. 

Improvement: For a school, positive change in: (1) the percentage of students passing 
an assessment; and (2) the number of students achieving CCR. 

Indicators: Measures of performance that are not student test scores. 

Norm referenced: A test in which an individual's performance is compared to the 
performances of other individuals in a peer group. 

Multiple measures: Multiple indicators and sources of evidence of student learning, of 
various kinds, gathered at multiple points in time. 

Performance: For a student, primarily determined by the student's score on one or 
more assessments. For a school, determined by the performance of all students, in 



addition to other established indicators (such as attendance, graduation rate, etc.). 

Targeted growth: the minimum growth expected for an individual student, as 
measured by performance on two consecutive assessments. 

Title I: A federal program that provides additional funding for schools with high poverty 
levels among students. Title I schools are subject to additional regulation and 
requirements by the federal government. 

Trajectory model: A model in which a student's growth towards a goal and future 
achievement of the goal is determined using at least two data points. 



APPENDIX B 



Simple Explanation of Indiana A-F Accountability System 

(From the "Examination of Indiana's A to F School Accountability Model", September 6, 
2013, prepared by John R. Grew and William J. Sheldrake, Appendix D) 



Elementary & Middle School (EMS) Model 



A School's grade is based on English/Language arts and math test results and various 
adjustments according to the following steps: 

1. Preliminary scores for both English/Language Arts (ELA) math tests are based on the 
percentage of a school's students that passed ISTEP+, IMAST and ISTAR. The 
preliminary score is determined using a proficiency grading scale awarding a grade for 
a given passage rate: 

90.0 - 100% = 4.00 points 70.0 - 74.9% - 2.00 points 

85.0 - 89.9% = 3.50 65.0 - 69.9% = 1 .50 

80.0 - 84.9% = 3.00 60.0 - 64.9% = 1 .00 

75.0 — 79.9% = 2.50 0.00 - 59.9% = 0.00 



2. A school's preliminary score for both ELA and math may be raised or lowered based 
on student academic growth: 

a. The preliminary score is raised by 1 .00 if at least 42.5% of the school's lowest 
performing students on ISTEP+ (the bottom 25%) score high growth on the ELA 
test and for the math test, at least 44.9% of students score high growth. 

b. The preliminary score is raised by 1 .00 if at least 36.2% of the school's 
remaining students on ISTEP+ (the top 75%) score high growth on the ELA test 
and for the math test, at least 36.2% of students score high growth. 

c. The preliminary score may be lowered by 1 .00 if 39.8% or more of all students 
taking ISTEP+ score low growth on the ELA test and for math test, less than 
42.4% of students score low growth. 

3. A school's score will also be lowered by 1 .00 if student participation in testing is: 

a. Less than 95% of their lowest performing students (bottom 25%) take ISTEP+. 

b. Less than 95% of their remaining students (top 75%) take ISTEP+, ISTAR, 
and IMAST. 



4. To determine the final grade for an EMS, sum the ELA and Math grades and divide 
by two. 

High School (HS) "A-F" Model 

The high school grade is determined by calculating scores on four weighted measures: 

1 . English 10 End of Course Assessment (ECA) - weighted at 30% 

2. Algebra I ECA - weighted at 30% 

3. Graduation Rate - weighted at 30% 

4. College & Career Readiness - weighted at 10% 

The steps in determining the high school score are as follows: 
1 & 2. Determining EnglishIO and Algebra I ECA Scores: 

a. Schools receive a preliminary score based on the percentage of their students 
in the 10th grade cohort that passed the ECA or ISTAR. The same proficiency 
grading scale (above) for EMS ELA and math is used to determine the 
preliminary score (e.g. a 90% passage rate = 4.00 points). 

b. The preliminary score is raised by 0.50 if there is at least a 10.3 percentage 
point improvement in the English passage rate and by 0.50 if there is at least a 
17.1% percentage point improvement in the math passage rate (from the 8th 
grade passage rates for ISTEP+, IMAST or ISTAR to the 10th grade ECA or 
ISTAR). 

c. The preliminary score is lowered by 0.50 if there is -0.1 percentage point or 
greater decline in the English or math passage rate (from the 8th grade ISTEP+, 
IMAST or ISTAR to the 1 0th grade ECA or ISTAR). 

d. The preliminary score shall be raised by 0.50 if at least 59.3% of students 
taking English or 62.8% of students taking math tests that did not pass the ECA 
or ISTAR in 10th grade do so by graduation. 

3. Determining the Graduation Rate Score: 

Schools receive a preliminary score based on their four-year graduation cohort rate. A 
proficiency grading scale (same as that used for EMS ELA and math) is used to 
determine the preliminary score (e.g. a 90% or higher passage rate = 4.00 points). For 
school years prior to 2014-15, the preliminary score is the final graduation rate score. 



Note: Beginning with the 2014-15 school year, the preliminary score will be adjusted as 
follows: 

a. The preliminary score is raised if 34.4% or more students receive non-waiver Honors 
Diplomas. 

b. The preliminary score is lowered if 32.8% or more students receive general or waiver 
diplomas. 

c. The preliminary score is raised if 13.2% of students that did not graduate within four 
years do so in five years. 

4. Determining the College & Career Readiness Score: 

Schools receive a score based on the percentage of graduates who receive at least one 
of the following: 

a. a passing score (3, 4, or 5) on an AP exam; or 

b. a passing score (4, 5, 6, or 7) on an IB exam; or 

c. three (3) verifiable college credits from the Priority Liberal Arts or CTE course 
lists; or 

d. a IDOE approved industry certification. 

The college and career readiness score is based on the percentage of students 
achieving one of the above activities using a different proficiency scale than above: 

25.0- 100% -4.00 points 5.0-11.6% = 1.00 points 

18.4 - 24.9% = 3.00 0.0 - 4.9% = 0.00 

11.7-18.3% = 2.00 

Determining a Final Grade for a High School 

The school's final grade is determined by summing the weighted scores from steps 1-4 
above. 



Determining a School Corporation Combined Score 

1. Determine the % of total school corporation students enrolled in EMS (grades 3-8) 
and H.S. (grades 9-12). 



2. Multiply the % of EMS students by the average grade for all EMS schools. 

3. Multiply the % of H.S. students by the average grade for all HS schools. 

4. Sum steps 2 and 3 to determine the combined score. 



SAMPLE Status Improvement Value Table 




Current Year Level 


Previous Year Level 


Did Not Pass-1 


Did Not Pass-2 


Did Not Pass-3 


Pass-1 


Pass-2 


Pass Plus-1 


Pass Plus-2 


Pass Plus-3 


Pass Plus-3 


15 


30 


45 


60 


75 


90 


105 


120 


Pass Plus-2 


25 


40 


55 


70 


85 


100 


115 


— — , — _ 

130 


Pass Plus -1 


35 


50 


65 


80 


95 


110 


125 


140 


Pass-2 


45 


60 


75 


90 


105 


120 


135 


150 


Pass-1 


55 


70 


85 


100 


115 


130 


145 


160 


Did Not Pass-3 


65 


80 


95 


110 


125 


140 


155 


170 


Did Not Pass-2 


75 


90 


105 


120 


135 


150 


165 


180 


Did Not Pass-1 


85 


100 


115 


130 


145 


160 


175 


190 
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Sample Growth: 
Students Meeting Targeted Growth 




2009 2010 2011 2012 

Students meeting minimum requirement slope 



Growth to Proficiency = ^^^^^^~"™"™™™^^^™ 

All students with 2 data points 



Indiana Department of Education 

2014-15 SAMPLE Accountability Report Card 
SAMPLE School of Indiana (1234) 



Component 


Gracles Oi-08 


Grades 09 - 11 


Grade 12 


Overall 


Performance 










English/Language Arts 
Math 


Participation 
Pass Rate Rate 




Enrollment 
Points Ratio 


Participation 
Pass Rate Rate 




Enrollment 
Points Ratio 


Rate 




Enrollment 
Points Ratio 


Pinal 

Points Weighting Points 
































































Reading 

CCR Achievement 

(IC,I8,DC,AP,PSAT)* 

Graduation 


































































































Total Performance Points: 0.000 0.000 


Growth 






English/Language Arts 

Math 

Reading 

CCR Achievement 
(IC,IB,DC,AP,PSAT) 
Graduation 


Percent 
Students 

Meeting Higher Lower 

Targeted Performing Performing Enrollment 
Growth (Categorical) (Categorical) Points Ratio 


Percent 
Students 

Meeting Higher Lower 

Targeted Performing Performing Enrollment 
Growth* (Categorical)' (Categorical)* Points Ratio 


Improvement 

Grade 10 to Improvement Enrollment 
Grade 12** Rate Points Ratio 


Pinal 

Points Weighting Points 














































































































































































*Retain Improvement 08 to 10 
during transition 

Total Growth Points: 0.000 0.000 


Performance: 
Growth: 
















Overall Points: 0.000 
Overall Grade: 
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